Re: [GENERAL] 8k limit - Mailing list pgsql-general

From Herouth Maoz
Subject Re: [GENERAL] 8k limit
Date
Msg-id l03110702b25c92e99153@[147.233.159.109]
Whole thread Raw
In response to Re: [GENERAL] 8k limit  (The Hermit Hacker <scrappy@hub.org>)
Responses Re: [GENERAL] 8k limit
List pgsql-general
At 4:15 +0200 on 28/10/98, The Hermit Hacker wrote:


> There are plans for doing what Bruce refers to as...what was his term?
> Its basically 'row wrapping'...the row limit is still 8k, but the server
> is smart enough to link two rows together to give you larger then that.
>
> It won't be in v6.4...but should be (or something similar) in a future
> release...

Here is an idea to solve the text problem.

Create a table for the bodies of the messages. Something like

CREATE TABLE bodies (
    mesg_id int4,
    chunk_no int4,
    chunk    text );

In the application, divide your text into chunks of, say, 6k. Suppose your
application is written in perl, you'd have an array of chunks @chunks

Then you write the headers of your message to the main mail table (the one
which has the headers). The message gets an ID from a sequence. You get the
value of that sequence.

Then you loop over the @chunks array, in each iteration inserting into the
bodies table the given message id, the loop iterator (as chunk_no), and the
content of the $chunks[$i] itself.

Then, in your app, if you want to reconstruct a message, you just retrieve
its headers (including mesg_id). Then you:

SELECT chunk_no, chunk
FROM bodies
WHERE mesg_id = <your message>
ORDER BY chunk_no;

Ignore the chunk_no in the returned set, and just concatenate all the
returned chunks in order. Then you have the body.

Advantage of this method: Unlike the large-objects interface,

(a) The text will be visible in psql in case you need to fix something.
(b) You will be able to do some rough searching in the form

    SELECT DISTINCT mesg_id
    FROM bodies
    WHERE chunk like '%something%';

(c) pg_dump will dump the table, so you needn't come up with an elaborate
    backup scheme of your own (pg_dump DOES NOT backup LOBs).

Herouth

--
Herouth Maoz, Internet developer.
Open University of Israel - Telem project
http://telem.openu.ac.il/~herutma



pgsql-general by date:

Previous
From: "SeeHyun Lee"
Date:
Subject: Vacuum problem.
Next
From: Lincoln Spiteri
Date:
Subject: Creating web images from postgres data